Cellular Neural Networks or Cellular Nonlinear Networks (CNN) have been applied to many different fields and problems including, but limited to, image processing since 1988. However, most of the prior art CNN approaches are either based on software solutions (e.g., Convolutional Neural Networks, Recurrent Neural Networks, etc.) or based on hardware that are designed for other purposes (e.g., graphic processing, general computation, etc.). As a result, CNN prior approaches are too slow in term of computational speed and/or too expensive thereby impractical for processing large amount of imagery data. The imagery data can be from any two-dimensional data (e.g., still photo, picture, a frame of a video stream, converted form of voice data, etc.).
In additional to convolutional layers, activation layers and pooling layers, ResNet requires operations of a pair of convolutional layers followed by element-wise add operations (i.e., a short path). It would therefore be desirable to implement deep neural network using 3×3 convolutional filter kernels to replace such operations in a CNN based integrated circuit.